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(57) ABSTRACT 

An apparatus and method is provided for converting digital 
images for use in an imaging system. The apparatus includes 
a data memory which stores digital data representing an 
image having a circular or spherical field of view such as an 
image captured by a fish-eye lens, a control input for receiv- 
ing a signal for selecting a portion of the image, and a con- 
verter responsive to the control input for converting digital 
data corresponding to the selected portion into digital data 
representing a planar image for subsequent display. Various 
methods include the steps of storing digital data representing 
an image having a circular or spherical field of view, selecting 
a portion of the image, and converting the stored digital data 
corresponding to the selected portion into digital data repre- 
senting a planar image for subsequent display. In various 
embodiments, the data converter and data conversion step 
may use an orthogonal set of transformation algorithms. 

1 Claim, 6 Drawing Sheets 
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OMNIVIEW MOTIONLESS CAMERA 
ORIENTATION SYSTEM 

This application is a continuation of application Ser. No. 
08/386,912, filed Feb. 8, 1995 now abandoned. 

This application is a continuation of U.S. application Ser. 
No. 08/339,663 filed Nov. 1 1, 1994 now abandoned, which is 
a continuation of U.S. application Ser. No. 08/189,585 filed 
Jan. 31, 1994 (now U.S. Pat. No. 5,384,588), which is a 
continuation-in-part of U.S. application Ser. No. 08/014,508 
filed Feb. 8, 1993 (now U.S. Pat. No. 5,359,363), which is a 
continuation-in-part of U.S. application Ser. No. 07/699,366 
filedMay 13, 1991 (now U.S. Pat. No. 5, 185,667). Each of the 
above-referenced U.S. patents is expressly incorporated by 
reference herein. 

This invention was made with Government support under 
contract NAS1-18855 awarded by NASA. The Government 
has certain rights in this invention. 

BACKGROUND OF THE INVENTION 

1 . Technical Field 

This invention relates generally to an apparatus and 
method for transforming perspective -distorted circular field 
of view images into non-distorted, normal perspective images 
having various orientations, rotations, and magnifications 
within the field of view. 

2. Related Information 

Camera viewing systems are used in abundance for sur- 
veillance, inspection, security, and remote sensing. Remote 
viewing is critical, for example, for robotic manipulation 
tasks. Close viewing is necessary for detailed manipulation 
tasks wile wide-angle viewing aids positioning of the robotic 
system to avoid collisions with the work space. Most of these 
systems use either a fixed-mount camera with a limited view- 
ing field to reduce distortion, or they utilize mechanical pan- 
and-tilt platforms and mechanized zoom lenses to orient the 
camera and magnify its image. In the applications where 
orientation of the camera and magnification of its image are 
required, the mechanical solution is large in size and can 
subtend a significant volume making the viewing system 
difficult to conceal or use in close quarters. Several cameras 
are usually necessary to provide wide-angle viewing of the 
work space. 

In order to provide a maximum amount of viewing cover- 
age or subtended angle, mechanical pan/tilt mechanisms usu- 
ally use motorized drives and gear mechanisms to manipulate 
the vertical and horizontal orientation. An example of such a 
device is shown in U.S. Pat. No. 4,728,839 issued to J. B. 
Coughlan, et al, on Mar. 1 , 1988. Collisions with the working 
environment caused by these mechanical pan/tilt orientation 
mechanisms can damage both the camera and the work space 
and impede the remote handling operation. Simultaneously, 
viewing in said remote environments is extremely important 
to the performance of inspection and manipulation activities. 

Camera viewing systems that use internal optics to provide 
wide viewing angles have also been developed in order to 
minimize the size and volume of the camera and the intrusion 
into the viewing area. These systems rely on the movement of 
either a mirror or prism to change the tilt-angle of orientation 
and provide mechanical rotation of the entire camera to 
change the pan angle of orientation. Additional lenses are 
used to minimize distortion. Using this means, the size of the 
camera orientation system can be minimized, but “blind 
spots” in the center of the view result. Also, these systems 
typically have no means of magnifying the image and or 
producing multiple images from a single camera. 
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References that may be relevant to the evaluation of the 
present invention are U.S. Pat. Nos. 4,772,942 issued to M. J. 
Tuck on Sep. 20, 1988; 5,023,725 issued to D. McCutchen on 
Jun. 11, 1991; 5,067,019 issued to R. D. Juday on Nov. 19, 
5 1991; and 5,068,735 issued to K. Tuchiya, et al on Nov. 26, 
1991. 

Accordingly, it is an object of the present invention to 
provide an apparatus that can provide an image of any portion 
of the viewing space within a selected field-of-view without 
10 moving the apparatus, and then electronically correct for 
visual distortions of the view. 

It is another object of the present invention to provide 
horizontal orientation (pan), vertical orientation (tilt) and 
rotational orientation (rotation) of the viewing direction with 
15 no moving mechanisms. 

It is another object of the present invention to provide the 
ability to magnify or scale the image (zoom in and out) 
electronically. 

It is another object of the present invention to provide 
20 electronic control of the image intensity (iris level). 

It is another object of the present invention to be able to 
accomplish pan, tilt, zoom, rotation, and iris adjustments with 
simple inputs made by a lay person from a joystick, keyboard 
controller, or computer controlled means. 

25 It is also an object of the present invention to provide 
accurate control of the absolute viewing direction and orien- 
tations using said input devices. 

A further object of the present invention is to provide the 
ability to produce multiple images with different orientations 
30 and magnifications simultaneously from a single input image. 

Another object of the present invention is to be able to 
provide these images at real-time video rates, e.g. thirty trans- 
formed images per second, and to support various display 
format standards such as the National Television Standards 
35 Committee RS-170 signal format and/or higher resolution 
formats currently under development. 

It is also an object of the present invention to provide a 
system that can be used for automatic or manual surveillance 
of selected environments, with optical views of these envi- 
40 ronments corrected electronically to remove distortion so as 
to facilitate this surveillance. 

These and other objects of the present invention will 
become apparent upon consideration of the drawings herein- 
after in combination with a complete description thereof. 

45 

DISCLOSURE OF THE INVENTION 

In accordance with the present invention, there is provided 
an omnidirectional viewing system that produces the equiva- 
50 lent of pan, tilt, zoom, and rotation within a selected field-of- 
view with no moving parts. Further, the present invention 
includes means for controlling this omnidirectional viewing 
in surveillance applications. This device includes a means for 
digitizing an incoming or prerecorded video image signal, 
55 transforming a portion of the video image based upon opera- 
tor or preselected commands, and producing one or more 
output images that are in correct perspective for human view- 
ing. In one embodiment, the incoming image is produced by 
a fisheye lens which has a wide angle field-of-view. This 
60 image is captured into an electronic memory buffer. A portion 
of the captured image, either in real time or as prerecorded, 
containing a region-of-interest is transformed into a perspec- 
tive correct image by an image processing computer. The 
image processing computer provides direct mapping of the 
65 image region-of-interest into a corrected image using an 
orthogonal set of transformation algorithms. The viewing 
orientation is designated by a command signal generated by 
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either a human operator or computerized input. The trans- 
formed image is deposited in a second electronic memory 
buffer where it is then manipulated to produce the output 
image or images as requested by the command signal. This is 
coupled with appropriate alarms and other outputs to provide 
a complete surveillance system for selected environments. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 shows a schematic block diagram of the signal 
processing portion of the present invention illustrating the 
major components thereof. 

FIG. 2 is an exemplary drawing of a typical fisheye image 
used as input by the present invention. Lenses having other 
field-of-view values will produce images with similar distor- 
tion, particularly when the field-of-view is about eighty 
degrees or greater. 

FIG. 3 is an exemplary drawing of the output image after 
correction for a desired image orientation and magnification 
within the original image. 

FIG. 4 is a schematic diagram of the fundamental geometry 
that the present invention embodies to accomplish the image 
transformation. 

FIG. 5 is a schematic diagram demonstrating the projection 
of the object plane and position vector into image plane 
coordinates. 

FIG. 6 is a block diagram of the present invention as uti- 
lized for surveillance/inspection applications incorporating 
the basic transformation of video images obtained with, for 
example, wide angle lenses to correct for optical distortions 
due to the lenses, together with the control of the surveillance/ 
inspection and appropriate alarm systems. 

FIGS. 7A and 7B, together, show a logic flow diagram 
illustrating one specific embodiment of controller operation 
for manual and automatic surveillance operations of the 
present invention. 

BEST MODE FOR CARRYING OUT THE 
INVENTION 

In order to minimize the size of the camera orientation 
system while maintaining the ability to zoom, a camera ori- 
entation system that utilizes electronic image transformations 
rather than mechanisms was developed. While numerous pat- 
ents on mechanical pan-and-tilt systems have been filed, no 
approach using strictly electronic transforms and wide angle 
optics is known to have been successfully implemented. In 
addition, the electro-optical approach utilized in the present 
invention allows multiple images to be extracted from the 
output of a single camera. These images can be then utilized 
to energize appropriate alarms, for example, as a specific 
application of the basic image transformation in connection 
with a surveillance system. As utilized herein, the term “sur- 
veillance” has a wide range including, but not limited to, 
determining ingress or egress from a selected environment. 
Further, the term “wide angle” as used herein means a field- 
of-view of about eighty degrees or greater. Motivation for this 
device came from viewing system requirements in remote 
handling applications where the operating envelop of the 
equipment is a significant constraint to task accomplishment. 

The principles of the optical transform utilized in the 
present invention can be understood by reference to the sys- 
tem 10 of FIG. 1 . (This is also set forth in the afore-cited U.S. 
patent application Ser. No. 07/699,366 that is incorporated 
herein by reference.) Shown schematically at 11 is a wide 
angle, e.g., a fisheye, lens that provides an image of the 
environment with a 180 degree field-of-view. The lens is 
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attached to a camera 12 which converts the optical image into 
an electrical signal. These signals are then digitized electroni- 
cally 13 and stored in an image buffer 14 within the present 
invention. An image processing system consisting of an 
5 X-MAP and a Y-MAP processor shown as 16 and 17, respec- 
tively, performs the two-dimensional transform mapping. 
The image transform processors are controlled by the micro- 
computer and control interface 15. The microcomputer con- 
trol interface provides initialization and transform parameter 
to calculation for the system. The control interface also deter- 
mines the desired transformation coefficients based on orien- 
tation angle, magnification, rotation, and light sensitivity 
input from an input means such as a joystick controller 22 or 
computer input means 23 . The transformed image is filtered 
15 by a 2-dimensional convolution filter 1 8 and the output of the 
filtered image is stored in an output image buffer 19. The 
output image buffer 19 is scanned out by display electronics 
20 to a video display device 21 for viewing. 

A range of lens types can be accommodated to support 
20 various fields of view. The lens optics 11 correspond directly 
with the mathematical coefficients used with the X-MAP and 
Y-MAP processors 16 and 17 to transform the image. The 
capability to pan and tilt the output image remains even 
though a different maximum field of view is provided with a 
25 different lens element. 

The invention can be realized by proper combination of a 
number of optical and electronic devices. The lens 11 is 
exemplified by any of a series of wide angle lenses from, for 
example, Nikon, particularly the 8 nun F2.8. Any video 
30 source 12 and image capturing device 13 that converts the 
optical image into electronic memory can serve as the input 
for the invention such as a Videk Digital Camera interfaced 
with Texas Instrument’ s TMS 3406 1 integrated circuits. Input 
and output image buffers 14 and 19 can be constructed using 
35 Texas Instrument TMS44C25 1 video random access memory 
chips or their equivalents. The control interface can be 
accomplished with any of a number of microcontrollers 
including the Intel 80C196. The X-MAP and Y-MAP trans- 
form processors 16 and 17 and image filtering 19 can be 
40 accomplished with application specific integrated circuits or 
other means as will be known to persons skilled in the art. The 
display driver can also be accomplished with integrated cir- 
cuits such as the Texas Instruments TMS34061. The output 
video signal can be of the NTSC RS-170, for example, com- 
45 patible with most commercial television displays in the 
United States. Remote control 22 and computer control 23 are 
accomplished via readily available switches and/or computer 
systems that also will be well known. These components 
function as a system to select a portion of the input image 
50 (fisheye or other wide angle) and then mathematically trans- 
form the image to provide the proper prospective for output. 
The keys to the success of the invention include: 

( 1 ) the entire input image need not be transformed, only the 
portion of interest; 

55 (2) the required mathematical transform is predictable 

based on the lens characteristics; and 

3) calibration coefficients can be modified by the end user 
to correct for any lens/camera combination supporting both 
new and retrofit applications. 

60 The transformation that occurs between the input memory 
buffer 14 and the output memory buffer 19, as controlled by 
the two coordinated transformation circuits 16 and 17, is 
better understood by referring to FIGS. 2 and 3. The image 
shown in FIG. 2 is a rendering of the image of a grid pattern 
65 produced by a fisheye lens. This image has a field-of-view of 
180 degrees and shows the contents of the environment 
throughout an entire hemisphere. Notice that the resulting 
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image in FIG. 2 is significantly distorted relative to human 
perception. Similar distortion will be obtained even with 
lesser field-of-view lenses. Vertical grid lines in the environ- 
ment appear in the image plane as 24 a, 24 b, and 24c. Hori- 
zontal grid lines in the environment appear in the image plane 
as 25 a, 25b, and 25c. The image of an object is exemplified by 
26. A portion of the image in FIG. 2 has been corrected, 
magnified, and rotated to produce the image shown in FIG. 3 . 
Item 27 shows the corrected representation of the object in the 
output display. The results shown in the image in FIG. 3 can 
be produced from any portion of the image of FIG. 2 using the 
present invention. The corrected perspective of the view is 
demonstrated by the straightening of the grid pattern dis- 
played in FIG. 3 . In the present invention, these transforma- 
tions can be performed at real-time video rates (e.g., thirty 
times per second), compatible with commercial video stan- 
dards. 

The transformation portion of the invention as described 
has the capability to pan and tilt the output image through the 
entire field of view of the lens element by changing the input 
means, e.g. the joystick or computer, to the controller. This 
allows a large area to be scanned for information as can be 
useful in security and surveillance applications. The image 
can also be rotated through any portion of 360 degrees on its 
axis changing the perceived vertical of the displayed image. 
This capability provides the ability to align the vertical image 
with the gravity vector to maintain a proper perspective in the 
image display regardless of the pan or tilt angle of the image. 
The invention also supports modifications in the magnifica- 
tion used to display the output image. This is commensurate 
with a zoom function that allows a change in the field of view 
of the output image. This function is extremely useful for 
inspection and surveillance operations. The magnitude of 
zoom provided is a function of the resolution of the input 
camera, the resolution of the output display, the clarity of the 
output display, and the amount of picture element (pixel) 
averaging that is used in a given display. The invention sup- 
ports all of these functions to provide capabilities associated 
with traditional mechanical pan (through 180 degrees), tilt 
(through 180 degrees), rotation (through 360 degrees), and 
zoom devices. The digital system also supports image inten- 
sity scaling that emulates the functionality of a mechanical 
iris by shifting the intensity of the displayed image based on 
commands from the user or an external computer. 

The postulates and equations that follow are based on the 
image transformation portion of the present invention utiliz- 
ing a wide angle lens as the optical element. These also apply 
to other field-of-view lens systems. There are two basic prop- 
erties and two basic postulates that describe the perfect wide 
angle lens system. The first property of such a lens is that the 
lens has a 2jt steradian field-of-view and the image it pro- 
duces is a circle. The second property is that all objects in the 
field-of-view are in focus, i.e. the perfect wide angle lens has 
an infinite depth-of-field. The two important postulates of this 
lens system (refer to FIGS. 4 and 5) are stated as follows: 

Postulate 1 : Azimuth Angle Invariability 

For object points that lie in a content plane that is perpen- 
dicular to the image plane and passes through the image plane 
origin, all such points are mapped as image points onto the 
line of intersection between the image plane and the content 
plane, i.e. along a radial line. The azimuth angle of the image 
points is therefore invariant to elevation and object distance 
changes within the content plane. 

Postulate 2: Equidistant Projection Rule 

The radial distance, r, from the image plane origin along 
the azimuth angle containing the projection of the object 
point is linearly proportional to the zenith angle (3, where (3 is 
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defined as the angle between a perpendicular line through the 
image plane origin and the line from the image plane origin to 
the object point. Thus the relationship: 

5 r=k|5 (1) 

Using these properties and postulates as the foundation of 
the lens system, the mathematical transformation for obtain- 
ing a perspective corrected image can be determined. FIG. 4 
shows the coordinate reference frames for the object plane 
and the image plane. The coordinates u,v describe object 
points within the object plane. The coordinates x,y,z describe 
points within the image coordinate frame of reference. 

The object plane shown in FIG. 4 is a typical region of 
1 5 interest to determine the mapping relationship onto the image 
plane to properly correct the object. The direction of view 
vector, DOV[x,y,z], determines the zenith and azimuth angles 
for mapping the object plane, UV, onto the image plane, XY. 
The object plane is defined to be perpendicular to the vector, 
20 DOV[x,y,z]. 

The location of the origin of the object plane in terms of the 
image plane [x,y,z] in spherical coordinates is given by: 

x=D sin p cos 3 
25 

y=D sin p sin 3 

z=D cos p (2) 

30 where D=scaler length from the image plane origin to the 
object plane origin, (3 is the zenith angle, and 3 is the azimuth 
angle in image plane spherical coordinates. The origin of 
object plane is represented as a vector using the components 
given in Equation 1 as: 

35 

DOV[x,y,z]=[D sin p cos 3,D sin p sin 3,D cos p] (3) 

DOV[x,y,z] is perpendicular to the object plane and its 
scaler magnitude D provides the distance to the object plane. 
By aligning the YZ plane with the direction of action of 
40 DOV[x,y,z], the azimuth angle 3 becomes either 90 or 270 
degrees and therefore the x component becomes zero result- 
ing in the DOV[x,y,z] coordinates: 

DOV[x,y,z]= /0, -D sin p ,D cos p J (4) 

45 Referring now to FIG. 5, the obj ect point relative to the UV 
plane origin in coordinates relative to the origin of the image 
plane is given by the following: 

x=u 

50 

y=v cos p 

z=v sin p (5) 

55 therefore, the coordinates of a point P(u,v) that lies in the 
object plane can be represented as a vector P[x,y,z] in image 
plane coordinates: 

P[x,y,z]=[u,v cos p,v sin p] (6) 

60 where P[x,y,z] describes the position of the object point in 
image coordinates relative to the origin of the UV plane. The 
object vector 0[x,y,z] that describes the object point in image 
coordinates is then given by: 

65 0[x,y,z]=DOVfx,y,z J +P[x,y,z] (7) 


0[x,y,z]=[u, v cos p -D sin p,v sin p +D cos p/ 


(8) 
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Projection onto a hemisphere of radius R attached to the 
image plane is determined by scaling the object vector 0[x, 
y,z] to produce a surface vector S[x,y,z]: 


S[x, y, z] 


RO[x, y, z ] 
\0[x, y, z\\ 


5 


(9) 


By substituting for the components of 0[x,y,z] from Equa- 10 
tion 8, the vector S[x,y,z] describing the image point mapping 
onto the hemisphere becomes: 


RO[u, (vcos B - Dsin/3), (vsin/5 + DcosjS)] (10) 15 

S[x, y, z] = = 

y u 2 + (vcos/5 - Dsin/3) 2 + (vsin/5 + DcosjS) 2 

The denominator in Equation 1 0 represents the length or 
absolute value of the vector 0[x,y,z] and can be simplified 20 
through algebraic and trigonometric manipulation to give: 


S[x, y, z ] 


RO[u, (vcos/5- Dsin/5), (vsin/3 + Dcos/?)] 
V w 2 + v 2 + D 2 


( 11 ) 


25 


From Equation 1 1 , the mapping onto the two-dimensional 
image plane can be obtained for both x and y as: 


30 


Ru (12) 

-v = . = 

V M 2 + V 2 + D 2 

R(vcosf3- Z)sin/f) (13) 35 

y = — 

V u 2 + v 2 + D 2 


Additionally, the image plane center to object plane dis- 
tance D can be represented in terms of the image circular 40 
radius R by the relation: 

D=mR (14) 

where m represents the scale factor in radial units R from 
the image plane origin to the object plane origin. Substituting 45 
Equation 14 into Equations 12 and 13 provides a means for 
obtaining an effective scaling operation or magnification 
which can be used to provide zoom operation. 


Ru 

(15) 

V« 2 + v 2 4 - m 2 R 2 


R(v cos/3 - mRsinfi) 

(16) 

Vw 2 + v 2 +m 2 R 2 



Using the equations for two-dimensional rotation of axes 
for both the UV object plane and the XY image plane the last 
two equations can be further manipulated to provide a more 
general set of equations that provides for rotation within the 
image plane and rotation within the object plane. 


R[uA - vB + m/?sin/3sind] 
V« 2 + v 2 +m 2 R 2 


(17) 


65 


-continued 

R[uC -vD- mRsinficosd ] ( 1 8) 

V« 2 + v 2 + m 2 R 2 

where: 

^4=(cos 0 cos 3-sin 0 sin 3 cos |3) 

2?=(sin 0 cos 3+cos 0 sin 3 cos |3) 

C=(cos 0 sin 3+sin 0 cos 3 cos |3) 

Z)=(sin 0 sin 3-cos 0 cos 3 cos |3) (19) 

and where: 

R=radius of the image circle 
p=zenith angle 

3=Azimuth angle in image plane 
0=Object plane rotation angle 
m=Magnification 
u,v=object plane coordinates 
x,y=image plane coordinates 

The Equations 1 7 and 1 8 provide a direct mapping from the 
UV space to the XY image space and are the fundamental 
mathematical result that supports the functioning of the 
present omnidirectional viewing system with no moving 
parts. By knowing the desired zenith, azimuth, and object 
plane rotation angles and the magnification, the locations of x 
and y in the imaging array can be determined. This approach 
provides a means to transform an image from the input video 
buffer to the output video buffer exactly. Also, the image 
system is completely symmetrical about the zenith, therefore, 
the vector assignments and resulting signs of various compo- 
nents can be chosen differently depending on the desired 
orientation of the object plane with respect to the image plane. 
In addition, these postulates and mathematical equations can 
be modified for various lens elements as necessary for the 
desired field-of-view coverage in a given application. 

The input means defines the zenith angle, p, the azimuth 
angle, 3, the object rotation, 0, and the magnification, m. 
These values are substituted into Equations 1 9 to determine 
values for substitution into Equations 17 and 18. The image 
circle radius, R, is a fixed value that is determined by the 
camera lens and element relationship. The variables u and v 
vary throughout the object plane determining the values for x 
and y in the image plane coordinates. 

From the foregoing, it can be seen that a wide angle lens 
provides a substantially hemispherical view that is captured 
by a camera. The image is then transformed into a corrected 
image at a desired pan, tilt, magnification, rotation, and focus 
based on the desired view as described by a control input. The 
image is then output to a television display with the perspec- 
tive corrected. Accordingly, no mechanical devices are 
required to attain this extensive analysis and presentation of 
the view of an environment through 180 degrees of pan, 180 
degrees of tilt, 360 degrees of rotation, and various degrees of 
zoom magnification. 

As indicated above, one application for the perspective 
correction of images obtained with a motionless wide angle 
camera is in the field of surveillance. The term “surveillance” 
is meant to include inspection and like operations as well. It is 
often desired to continuously or periodically view a selected 
environment to determine activity in that environment. The 
term “environment” is meant to include such areas as rooms, 
warehouses, parks and the like. This activity might be, for 
example, ingress or egress of some object relative to that 
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environment. It might also be some action that is taking place 
in that environment. It may be desired to carry out this sur- 
veillance either automatically at the desired frequency (or 
continuously), or upon demand by an operator. The size of the 
environment may require more than one motionless camera 5 
for complete surveillance. 

Such a surveillance system is indicated generally at 30 of 
FIG. 6. A video camera unit 32, including a wide angle lens 
31, is utilized to view the selected environment (or portion of 
the environment), with the output therefrom being electrical 10 
signals related to the elements as seen by the camera system. 
These signals, when present, are either directly presented to 
an image transformation system 33 (the components of FIG. 

1 without the camera/lens and the TV monitor thereof) or to a 
videotape recorder 3 4 for subsequent processing in the image 1 5 
transformation system 33. This permits evaluation during 
“post event” review as well as a review of events that occur in 
real time. It will be understood that additional camera-lens 
units in an environment, as well as videotape recorders, can 
be utilized as indicated at 35. 20 

Various external elements are utilized to govern the opera- 
tion of the transformation system. For example, appropriate 
discrete switches 36 are used for the selection of the environ- 
ment or portion of the environment to be monitored. These 
switches can be po sitioned (and operated) either at the control 25 
center or at the environment (or other remote location). When 
positioned in the environment, these switches can indicate 
some action occurring in the environment (door opening, 
window breaking, etc.) with the result that the virtual camera 
of the system is directed to the point of interest and then signal 30 
an external alarm for creating an audible alarm if desired. 
Also, alarm conditions can activate the video tape recorder 
discussed below. Since the system monitors the presence of 
an incoming video signal, the device can signal an alarm 
when the incoming video signal is disrupted. Where the 35 
monitoring is to be preselected, one input can be a computer 
38. Another form of control is through the use of operator 
controls 40 such that the operator can select at any time the 
operation of the transformation. Options that are available in 
either of these types of control are “Quad display” (either 40 
through the control by the computer 38 or the operator con- 
trols 40) wherein four displays occur on a monitor. Another 
option available through either control is that of “tweening” 
which is a selection of moving the effective view of the 
camera incrementally between active points or switching 45 
between active cameras within the environment. As previ- 
ously described, these inputs are used also for selecting pan, 
tilt, zoom and rotation. 

The output of the transformation system 33 is typically in 
digital format. As such, this output can control alarm enun- 50 
ciators 42 positioned at any location, or other forms of dis- 
crete alarms 44. They can also activate a videotape recording 
machine 46. In addition, the alarms 44 can be used to detect 
and announce loss of video signal, and permit external inter- 
rogation (manual or automated) of system status by the com- 55 
puter interface of the system parameters including compo- 
nent or power failure. Such interrogation would include 
verification of operation, video input, pan-tilt-rotation angles, 
magnification and setup parameters. As in the system of FIG. 

1, this surveillance system 30 provides for pictorial environ- 60 
ment display on a TV-type monitor 48 and/or on the tape of 
the recording machine 46. 

FIGS. 7A and 7A, jointly, form a logic flow diagram that 
illustrates how one specific embodiment of the controller 30 
can perform manual and automatic surveillance activities. A 65 
decision is made at 50 as to whether the system is under 
computer (external) control (see 38 of FIG. 6) or manual 
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(internal) control. If under computer operation, the camera 
orientations and magnifications are communicated directly to 
the system for action at 52. In the event of internal control, it 
is next determined if any environmental switches are closed 
as at 54. These switches typically are hard wired, magnetic 
infrared or other forms that indicate a change in the environ- 
ment in a certain location. The choice of a specific type of 
switch for each application will be known by persons skilled 
in the art. These changes (if “YES”) give rise to signals at 56 
to point its visual camera in the direction of interest and then 
signal an external alarm for creating an audible alarm 42 
and/or turning on the video tape recorder 34. 

After initiating these steps, or if the answer is “NO” at 54, 
the switches on the unit’s control panel are read at 58 to 
determine the configuration and display actions needed. 
“Quad display” (either four displays or one display on the 
monitor 48) is checked at 60 and, if the four displays are 
desired (the “YES”), this is initiated at 62. If “tweening” 
(incremental effective movement of a camera or switching 
between cameras) is desired, this is checked at 64 and the 
appropriate selection is made at 66. 

Inputs for pan, tilt, zoom and rotation are interpreted at 68 
and applied to the presently active display camera. Every user 
interaction resets the scan timer at 70 so that while the user is 
in control, no virtual camera change is occurring. When the 
scan time reaches zero, as monitored at 72, the next camera is 
made active and the image being displayed changes direction 
and/or content as at 74 to thereby update operation as at 52. 

From the foregoing, it will be understood by those versed in 
the art that an advanced surveillance system has been pro- 
vided. This system utilizes at least one motionless video 
camera, having a wide angle lens, within the environment 
where surveillance is desired. The perspective of images 
obtained with the lens/ camera are corrected through transfor- 
mation according to the technology of Ser. No. 07/699,366 
either directly or after storage in a videotape recorder. Many 
operational conditions are selectable including tilt, pan, zoom 
and rotation. Further, multi -image displays can be obtained, 
and the images can be incrementally scanned or switching 
between cameras are other options. The system provides for 
automatic operation coupled with user operation if desired. 

While certain specific elements of construction are indi- 
cated throughout the description of the present invention, 
these are given for illustration and not for limitation. Thus, the 
invention is to be limited only by the appended claims and 
their equivalents. 

The invention claimed is: 

1. A memory for a signal processor, comprising: 
a data structure, responsive to a control input representing 
a selection of a portion of an image stored in said 
memory, wherein said selection is chosen across a field 
of view, said data structure representing an orthogonal 
set of transformation algorithms; and 
a buffer memory adapted to store digital image data for 
transformation; 

wherein said data structure transforms data according to 
the following equations: 


R[uA -vB + mRsinfisind] 

V (w 2 + v 2 +m 2 R 2 ) 

R[uC -vD- mflsin/fcosd] 

V (w 2 + v 2 + m 2 R 2 ) 
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where: 

and where: 

A =( cos 0 cos 3-sin 0 sin 3 cos p) 

R=radius of the image circle 
p=zenith angle 

3=Azimuth angle in image plane 

2?=(sin 0 cos 3+cos 0 sin 3 cos p) 

5 0=object plane rotation angle 

m=Magnification 

C=(cos 0 sin 3+sin 0 cos 3 cos p) 

u,v=object plane coordinates 
x,y=image plane coordinates. 

£)=(sin 0 sin 3-cos 0 cos 3 cos p) 

* * * * * 



